Defect identification using machine learning in an additive manufacturing system

ABSTRACT

An additive manufacturing system comprises an apparatus arranged to distribute layer of metallic powder across a build plane and a power source arranged to emit a beam of energy at the build plane and fuse the metallic powder into a portion of a part. The system includes a processor configured to steer the beam of energy across the build plane and receive data generated by one or more sensors that detect electromagnetic energy emitted from the build plane when the beam of energy fuses the metallic powder. The received data is converted into one or more parameters that indicate one or more conditions at the build plane while the beam of energy fuses the metallic powder. The one or more parameters are used as input into a machine learning algorithm to detect one or more defects in the fused metallic powder.

CROSS-REFERENCES TO OTHER APPLICATIONS

This application claims priority to U.S. provisional patent applicationSer. No. 63/062,949, for “DEFECT IDENTIFICATION USING MACHINE LEARNINGIN AN ADDITIVE MANUFACTURING SYSTEM” filed on Aug. 7, 2020 which ishereby incorporated by reference in entirety for all purposes.

BACKGROUND OF THE INVENTION

Additive manufacturing, or the sequential assembly or construction of apart through the combination of material addition and applied energy,takes on many forms and currently exists in numerous implementations andembodiments. Additive manufacturing can be carried out by using any of anumber of various processes that involve the formation of a 3-D part ofany shape. The various processes that are used for making metallic partshave in common the sintering and/or melting of powdered or granular rawmaterial, layer by layer using one or more high power energy sourcessuch as a laser or electron beam. The generation of defects in thefinished part can be caused by multiple sources and such defects can bedifficult to track and/or to detect. The defects can result indegradation of material properties of the finished part leading topremature failure. New methods to detect and characterize defects inparts that are made using additive manufacturing systems are needed.

SUMMARY

Some embodiments of the present disclosure relate to additivemanufacturing systems that employ machine learning to detect andidentify different types of in-process defects that occur during themanufacturing process. Some embodiments relate to methods of training amachine learning algorithm with known defective parts. Other embodimentsrelate to methods of training the machine learning algorithm todistinguish one type of defect from another.

In some embodiments an additive manufacturing system comprises anapparatus arranged to distribute layer of metallic powder across a buildplane. A power source is arranged to emit a beam of energy at the buildplane and fuse the metallic powder into a portion of a part. A processoris configured to steer the beam of energy across the build plane and toreceive data generated by one or more sensors that detectelectromagnetic energy emitted from the build plane when the beam ofenergy fuses the metallic powder. The processor converts the receiveddata into one or more parameters that indicate one or more conditions atthe build plane while the beam of energy fuses the metallic powder anduses the one or more parameters as input into a machine learningalgorithm to detect one or more defects in the fused metallic powder.

In some embodiments the machine learning algorithm is further configuredto determine a type of the one or more defects. In various embodimentsthe type of the one or more defects includes a lack of fusion defect, aporosity defect or an inclusion defect. In some embodiments the one ormore parameters includes determining a thermal emission density (TED)that includes measuring an amount of energy radiated from the buildplane during one or more scans and determining area of the build planetraversed during the one or more scans. In various embodiments the oneor more parameters includes TEP that includes identifying spectral peaksassociated with material properties of a batch of powder and selecting afirst wavelength and a second wavelength spaced apart from the firstwavelength, and determining an amount of energy radiated from the buildplane based upon a ratio of energy radiated at the first wavelength toenergy radiated at the second wavelength.

In some embodiments the machine learning algorithm includes one or moretraining parameters based on a known-defective part. In variousembodiments the one or more training parameters includes a void. In someembodiments the one or more training parameters includes an inclusion.In various embodiments the one or more sensors includes an on-axisphotodetector.

In some embodiments an additive manufacturing system comprises an energysource arranged to fuse metallic powder, a sensor arranged to detectelectromagnetic energy emitted during the fusing of the metallic powderand a processor that receives data from the sensor and employs a trainedmachine learning algorithm to detect a defect in the fused metallicpowder. In various embodiments the trained machine learning algorithmdetermines a type of the defect. In some embodiments the type of defectincludes a lack of fusion of the metallic powder defect, a porositydefect or an inclusion defect. In various embodiments the processor isconfigured to calculate one or more parameters based at least in part onthe electromagnetic energy.

In some embodiments the one or more parameters includes determining athermal emission density (TED) that includes measuring an amount ofenergy radiated from the fused metallic powder during one or more scansof the energy source and determining area of a build plane traversedduring the one or more scans. In various embodiments the one or moreparameters includes identifying spectral peaks associated with materialproperties of a batch of the metallic powder and selecting a firstwavelength and a second wavelength spaced apart from the firstwavelength, and determining an amount of energy radiated from a buildplane based upon a ratio of energy radiated at the first wavelength toenergy radiated at the second wavelength.

In some embodiments the one or more training parameters are derived froma known-defective part having at least one region having lack of fusionof the metallic powder. In various embodiments the one or more trainingparameters are derived from a known-defective part having at least oneinclusion. In some embodiments the one or more training parameters arederived from a known-defective part having at least one region havingporosity. In various embodiments the one or more sensors includes anon-axis photodetector. In some embodiments the trained machine learningalgorithm is based on a random forest model.

Numerous benefits are achieved by way of the present invention overconventional techniques. For example, embodiments of the presentinvention provide the ability to detect defects in-process and correctthem or stop the build of the part resulting in significant improvementsin manufacturing efficiency. Embodiments of the present invention alsoprovide the ability of defect identification so an operator can make amore informed decision during the build to repair the part ordiscontinue building the part.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1A illustrates a simplified diagram of a machine learning algorithmthat can be used to detect and identify defects generated by an additivemanufacturing system, according to embodiments of the disclosure;

FIG. 1B describes a training process that can be used for the machinelearning algorithm described in FIG. 1A;

FIG. 1C describes an example process of employing a machine learningalgorithm described in FIG. 1A;

FIG. 2A illustrates an isometric image of an example training build thatis designed to exhibit four common types of part anomalies, according toembodiments of the disclosure;

FIG. 2B shows a top down visualization of one of the control parts usedin the training build of FIG. 2A;

FIG. 3 shows tessellations of CT data for lack of fusion parts used inthe training build of FIG. 2A;

FIG. 4 shows tessellations of the CT data for gas porosity parts used inthe training build of FIG. 2A;

FIG. 5 shows tessellations of the CT data for keyhole parts used in thetraining build of FIG. 2A;

FIG. 6 shows tessellations of the CT data for inclusion parts used inthe training build of FIG. 2A;

FIG. 7 illustrates an example heat map that can be used to diagnose howwell model prediction images compare to training labels, according toembodiments of the disclosure;

FIG. 8A shows lack of fusion anomalies in a CT visualization, accordingto embodiments of the disclosure;

FIG. 8B shows a machine learning layer diagnostic visualization usingthe heatmap of FIG. 7 ;

FIG. 9A shows the gas porosity anomalies in the CT visualization,according to embodiments of the disclosure;

FIG. 9B shows the machine learning layer diagnostic visualization usingthe heatmap of FIG. 7 ;

FIG. 10A shows the keyhole anomalies in the CT visualization, accordingto embodiments of the disclosure;

FIG. 10B shows the machine learning prediction diagnostic visualizationusing the heatmap of FIG. 7 ;

FIG. 11A shows the tungsten inclusion anomalies in the CT visualization,according to embodiments of the disclosure;

FIG. 11B shows the machine learning prediction diagnostic visualizationusing the heatmap of FIG. 7 ;

FIG. 12A shows the test layer visualization with respect to keyholedefects, according to embodiments of the disclosure;

FIG. 12B shows the test layer visualization with respect to gasporosity, according to embodiments of the disclosure;

FIG. 12C shows the test layer visualization with respect to inclusions1210, according to embodiments of the disclosure;

FIG. 12D shows the lack of fusion model prediction on a lack of fusionpart, according to embodiments of the disclosure;

FIGS. 12E-12G show the lack of fusion model predictions on parts withkeyhole, gas porosity and tungsten anomalies, respectively, according toembodiments of the disclosure;

FIG. 12H shows the prediction on PrintRite3D® visualization generated bythe lack of fusion model, according to embodiments of the disclosure;

FIG. 13A shows the test layer visualization with respect to lack offusion, according to embodiments of the disclosure;

FIGS. 13B and 13C show the test layer visualization with respect tokeyhole and tungsten inclusion, respectively, according to embodimentsof the disclosure;

FIGS. 13D, 13E, 13F and 13G show the model predictions on parts withlack of fusion, gas porosity, keyhole, and tungsten inclusion anomalies,respectively, according to embodiments of the disclosure;

FIG. 13H shows the prediction on PrintRite3D® visualization generated bythe GP model, according to embodiments of the disclosure;

FIG. 14A shows the test layer visualization with respect to lack offusion, according to embodiments of the disclosure;

FIGS. 14B and 14C show the test layer visualization with respect to gasporosity, and tungsten anomalies, respectively, according to embodimentsof the disclosure;

FIG. 14D shows the KH model prediction on a keyhole part, according toembodiments of the disclosure;

FIGS. 14E-14G show the model predictions on the parts with lack offusion, gas porosity and tungsten anomalies, respectively, according toembodiments of the disclosure;

FIG. 14H shows the prediction on PrintRite3D® visualization generated bythe KH model, according to embodiments of the disclosure;

FIGS. 15A-15C show the test layer visualization with respect to lack offusion, gas porosity, and keyhole anomalies, respectively, according toembodiments of the disclosure;

FIGS. 15D, 15E, 15F and 15G show the model predictions on the parts withlack of fusion, tungsten inclusion, gas porosity and keyhole anomalies,respectively, according to embodiments of the disclosure;

FIG. 15H shows the prediction on PrintRite3D® visualization generated bythe tungsten inclusion model, according to embodiments of thedisclosure;

FIG. 16 shows an example additive manufacturing system, according toembodiments of the disclosure; and

FIG. 17 shows an expanded block diagram of the example computing systemshown in FIG. 16 .

DETAILED DESCRIPTION

Some embodiments of the present disclosure relate to methods ofdetecting defects in a part that is made using an additive materialmanufacturing system. While the present disclosure can be useful for awide variety of configurations, some embodiments of the disclosure areparticularly useful for using one or more in-process metrics along withmachine learning to detect and identify the type of defect within partsmade by an additive manufacturing system, as described in more detailbelow.

Three-dimensional defect morphology and its relationship with in-processdata generated by an additive material manufacturing system can bedifficult to be interpreted by humans. Even when multiple metrics aregenerated from the in-process data it can be difficult for an operatorto use them to predict where anomalies are likely to have occurred in apart and even more difficult to predict the type of defect thatoccurred. More specifically a human operator may have difficultyextracting subtle changes within a particular metric or in relationshipsbetween multiple metrics that indicate a defect has occurred and/orindicate the type of defect that has occurred. Conversely, machinelearning algorithms can be effectively trained with known good parts andknown bad parts to effectively recognize these subtle changes and detectthat a defect has occurred. Further, parts having a known type of defectsuch as lack of fusion, porosity, keyhole and contamination can be usedto train the machine learning algorithm to identify a particular type ofdefect. With the added knowledge of the type of defect that hasoccurred, a more informed decision can be made as to whether the partshould continue in the build process, be rejected or be repaired. Thissignificantly improves the cost and efficiency of additive manufacturingsystems as compared to sending samples for external analysis.

FIG. 1A illustrates a visual summary of one embodiment of a machinelearning algorithm 100 that can be used to detect and identify defectsgenerated by an additive manufacturing system. As shown in FIG. 1 ,algorithm 100 is broken up into a training process 105 and a qualityassurance process 110. Starting first with the training process,training builds of parts manufactured with the additive manufacturingsystem are performed. During the manufacturing, in process metrics,labeled as Sigma Metrics in FIG. 1 , are collected. In some embodimentsthese training builds can have induced defects such that known defectsare introduced into the parts and can be used to train the machinelearning algorithm, explained in greater detail below. The parts havingknown defects can then be analyzed with either non-destructive ordestructive tests to verify and map the location of the defects. Thisdata can be overlaid with the in-process metrics and used as “trainingdata” to train the machine learning algorithm. Training process 105 isdescribed in more detail in FIG. 1B.

As described herein, a machine learning algorithm is an application ofartificial intelligence that provides systems the ability toautomatically learn and improve from experience without being explicitlyprogrammed. Typically, machine learning algorithms build a mathematicalmodel based on sample data, often called “training data”, to makepredictions or decisions without being explicitly programmed to do so.

In some embodiments different part geometries, different materials, etc.can be used to create different machine learning algorithms that can betailored to specific parts, specific materials or other variables. Inother embodiments generic machine learning algorithms can be developedthat can handle varied part geometries and materials.

Now transitioning to the quality assurance process 110 in FIG. 1 , amachine learning algorithm can be selected from a model warehouse.Production builds can be initiated and in process metrics, labeled asSigma Metrics in FIG. 1 can be collected during the build process. Thesemetrics can be used as an input into the machine learning algorithm, andthe machine learning algorithm can output real-time data detectingand/or identifying defects in the production builds. As shown in FIG. 1the output from the machine learning algorithm can be displayed to auser via a “Part Quality Decision Dashboard” or through a“Visualization” user interface. As would be appreciated by one of skillin the art having the benefit of this disclosure, various other methodsof displaying and using the output of the machine learning algorithm canbe used and are within the scope of this disclosure. Thus, by employinga trained machine learning algorithm, real-time defect analysis and/oridentification can be performed, and an operator, or the additivemanufacturing system can make decisions with regard to the defectiveparts. In one example the system can take action to repair the defect,while in another example the system can cease building the defectiveparts to increase throughput and minimize waste. In further embodimentsthe defective parts can be identified and either rejected or analyzedwith a post-process system such as computed tomography. Qualityassurance process 110 is described in more detail in FIG. 1C.

In this example, in-process metrics that are used both for training andfor real-time metrics analyzed by the machine learning algorithm duringproduction builds are thermal energy density (TED), TED Sigma, thermalenergy Planck (TEP) and TEP Sigma. TED is a thermal energy densitymetric that indicates the thermal energy within a given area and TEP isa metric that evaluates a temperature of the plume using two or moreblackbody radiation frequencies. These and other in-process metrics aredescribed in U.S. Pat. Nos. 10,639,745 (TEP) and 10,479,020 (TED) and inU.S. patent application Ser. No. 16/182,478 (peak temperature), whichare incorporated by reference herein in their entirety for all purposes.As would be appreciated by one of skill in the art having the benefit ofthis disclosure, any in-process metrics can be used as input to themachine learning algorithm and are within the scope of this disclosure.

FIG. 1B describes one example training process 105 for a machinelearning algorithm, according to embodiments of the disclosure. Asdescribed above, the machine learning algorithm can be specificallydesigned for a particular part geometry, a particular material or anyother particular parameter. For example, a specific algorithm may betrained for an aluminum based material and a separate specific algorithmmay be trained for a tungsten based material because of thesignificantly different process temperatures involved and/or thedifferent emission characteristics of the material during the fusionprocess. In another embodiment a generic algorithm can be trained thatis effective for various different steel alloys or for parts having aparticular geometry.

As shown in FIG. 1B, training process 105 includes a first step 114 inwhich parts are selected for training. In some embodiments, for examplewhen training a generic algorithm, various different part geometries maybe selected, however in other embodiments the same part may be selectedfor the training process.

In step 116 a machine learning model is selected for the training. Insome embodiments the machine learning model can include one or more ofthe following models: Linear Regression, Logistic Regression, DecisionTree, SVM, Naive Bayes, kNN, K-Means, Random Forest, Extremely RandomForest, Dimensionality Reduction Algorithms, Gradient Boostingalgorithms, GBM, XGBoost, LightGBM, CatBoost or any other suitablemodel.

In step 118 parts are built by the additive manufacturing system. Insome embodiments the build includes both parts that incorporate knowndefects and parts that have no defects. More specifically, in someembodiments known defects such as embedded keyhole and lack of fusionanomalies can be induced in the parts by programming the energy sourceto emit a reduced power in these regions. Parts with induced gasporosity can be built by introducing anhydrous sodium acetate. Partswith inclusions can be built by adding tungsten powder. These areexamples of how one or more defects can be introduced into parts,however other embodiments may use different methods to introducedefects.

In step 120 while the parts in step 118 are being built by the additivemanufacturing system, one or more parameters are recorded. Morespecifically, in some embodiments one or more electromagnetic sensorsare used to record electromagnetic emission data from the weld poolduring fabrication of the parts. The data can be transformed into one ormore parameters that can be representative of conditions of the weldpool. For example, in some embodiments the parameters are TED or TEPwhich are explained in more detail below. In other embodimentstemperature, temperature variations, changes in temperature with time,etc. can comprise the electromagnetic emission data. In some embodimentsthe one or more parameters are associated with build coordinates of eachpart, thus the one or more parameters are “mapped” to each part in thebuild.

In step 122, after the build is completed the parts are analyzed and anydefects are “mapped”. More specifically, in some embodimentscomputer-aided tomography (CT) is used to evaluate each part and map theone or more defects onto the part geometry.

In step 124 the machine learning model is trained to create a trainedmachine learning algorithm. More specifically, in some embodiments theone or more mapped parameters are loaded into the model and overlaidwith corresponding mapped defect data from the CT evaluation. Thus themodel can compare regions of the parts where defects occur with regionsof the parts where defects do not occur and determine subtle shifts inthe mapped parameters, or shifts in relationships between the mappedparameters to identify defect signatures. Further, the model can betrained with different types of defects. For example the model can betrained to recognize a keyhole defect, then improved to differentiate akeyhole defect from a gas porosity defect.

In step 126 the trained machine learning algorithm can be saved and usedas either a generic algorithm or a specific algorithm for use in futureproduction builds. It will be appreciated that process 105 isillustrative and that variations and modifications are possible. Stepsdescribed as sequential may be executed in parallel, order of steps maybe varied, and steps may be modified, combined, added or omitted.

As shown in FIG. 1C, a training process such as training process 105 canbe employed during a production build for in-process quality control asshown in quality assurance process 110. In some embodiments trainingprocess 105 can be performed by a computing system that controls anadditive manufacturing system. In some embodiments the computing systemmay include one or more processors, as further described in FIG. 17 .

In step 130 a trained machine learning algorithm from training process105 can be selected. In some embodiments a generic algorithm can be usedwhile in other embodiments a specific algorithm that is suited for aparticular part geometry, material or other parameter is selected.

In step 132 the parameters for building the production parts are loadedinto the system and the build process is initiated. In some embodimentsthe loaded parameters include part geometry, for example in an STL orsimilar file format.

In step 134 the computing system steers a beam of energy across a workregion fusing metallic powder together in sequential layers to form oneor more production parts according to the loaded parameters.

In step 136 one or more electromagnetic sensors record electromagneticdata from the weld pool generated by the beam of energy fusing themetallic powder together. The system maps the data to the part geometry(e.g., to the STL file coordinates).

In step 138 the system coverts the received data from theelectromagnetic sensors to one or more parameters. In one example thereceived data is in the form of a variable voltage received from apyrometer. The variable voltage is transformed into a time varyingtemperature that is mapped onto the parts (e.g., onto the STL filecoordinates). In another embodiment the received data is converted intoa TED and/or a TEP parameter, which are explained in more detail below.Other suitable parameters may be derived, recorded and mapped onto theparts.

In step 140 the one or more parameters are used as input to the trainedmachine learning algorithm. The machine learning algorithm compares theparameters with its learned algorithm and generates an output.

In step 142 the output of the trained machine learning algorithm is usedto detect and identify defects within the one or more parts of thebuild. More specifically, in some embodiments the electromagnetic datais generated real-time, the one or more parameters are immediatelygenerated and input into the machine learning algorithm so defects canbe detected real-time as the part is being built. Further, in someembodiments the type of defect can be detected real-time.

In step 144 the system determines if the type of defect can be remedied.In some embodiments for example, the type of defect is determined to belack of fusion and is mapped to specific coordinates on the part. Thesystem can determine that this type of defect can be remedied and canproceed to step 146 and steer the energy beam back across that region tofuse the material. In another embodiment a new layer of powder can bedeposited and the system can increase the power while fusing that regionto fuse the underlying layer and remedy the defect. In step 148 theoperator can be notified of the remedy. In other embodiments the systemmay stop and request authorization from the operator before executingthe remedy algorithm.

Alternatively in step 144 the system may determine that the defectcannot be remedied and may proceed to step 150 which stops the buildprocess for the part or parts that include the defect. In step 152 theoperator can be notified and can make a decision to completely stopbuilding the defective parts to improve the efficiency of the system orto remedy the defects and continue building. It will be appreciated thatprocess 110 is illustrative and that variations and modifications arepossible. Steps described as sequential may be executed in parallel,order of steps may be varied, and steps may be modified, combined, addedor omitted.

The following portion of the disclosure describes an experimental setupthat can be used to train and test one embodiment of a machine learningalgorithm for an additive manufacturing system. Other embodiments canuse other suitable processes to train and employ machine learningalgorithms for detection and/or identification of part defects.

Training Setup

FIG. 2A illustrates an isometric image of an example training build 200that is designed to exhibit four common types of part anomalies. Thetraining build can be performed on any suitable additive manufacturingsystem. In one example embodiment it can be performed on an EOS M290equipped with Sigma Labs PrintRite3D® system. In this particularexample, metallic powder from PraxAir was used and is designated asTitanium Ti-6Al-4V alloy powder with a particle size distribution of23-50 microns. Training build consists of five separate regions. At thefar right a nominal control group 205 is shown. Two parts each ofembedded keyhole 210 and lack of fusion 215 which include sphericalvoids ranging from 1.00 mm-0.050 mm diameter spheres. These anomaliescan be induced in the parts by programming the energy source to emit areduced power in these regions. Fourth was a set of parts with inducedgas porosity 220 due to the introduction of anhydrous sodium acetate atthree discrete layer heights at an amount per layer of 1.0 g, 0.5 g and0.25 g. Four inclusion defect parts 225 were created by adding tungstenpowder with a particle distribution of 20-50 microns at the same 3-layerheights with an amount per layer of 1.0 g, 0.5 g and 0.25 g. All mainbody scans were standard EOS parameters. FIG. 2B shows a top downPrintRite3D® visualization of one of the control parts 205 a.

CT Analysis

Computed Tomography (CT) was used as the ground truth for generatingtraining labels to be used in a supervised machine learning algorithm.After running the build, the PrintRite3D® in-process metrics wereexamined and one part for each anomaly type was chosen based on metricactivity. The physical specimens and Standard Tessellation Language(STL) files giving the build specifications were converted to STL filesshowing the location and morphology of detected anomalies registered tothe same coordinates as the STL specifications.

Lack of Fusion

FIG. 3 shows tessellations of the CT data for lack of fusion parts 215.As shown in FIG. 3 , when too little energy density is delivered to apart location the powder can fail to fuse. To create lack of fusionanomalies, the M290 printer was programmed to use low energy parametersto produce spheres of lack of fusion anomalies. The array of regions 217corresponds to the programmed lack of fusion spheres with decreasingsize toward the top. Groups of six spheres were arranged in a hexagonalpattern and ten regions with decreasing sphere diameter were laid out atdifferent heights in each part. The sphere diameters ranged from 0.9 mmto 0.05 mm. One part is for inclusions which was not included in theanalysis. The second part is for porosities, which are more appropriatefor describing the lack of fusion anomalies.

Gas Porosity

FIG. 4 shows tessellations of the CT data for gas porosity parts 220,with regions 223 corresponding to regions of porosity. There are severalmechanisms that can cause microstructural porosity in additivemanufacturing. In this experiment trapped gas was emulated by dosing thepart with foreign particles with low boiling point and high vaporpressure with the expectation that the evaporated particles would leavesmall pores in the product. The M290 printer was paused before spreadingthe low boiling point powder at three different layers. The buildchamber was opened, and sizes of 1 g, 0.5 g and 0.25 g of anhydroussodium acetate CH3COONa powder were deposited in the four parts. Of thefour gas porosity parts, the part at the top of the plate was selectedfor CT analysis because the in-process metrics showed stronger signalsat the programmed anomalies. The Metrology Center reported the anomaliesfor this part in an STL files. FIG. 4 shows tessellations of the voidsbecause of the contaminant particle injections from CT data distributedin three layers of a gas porosity part.

Keyhole

FIG. 5 shows tessellations of the CT data for keyhole parts 210 withregions 213 corresponding to keyhole regions. When too much laser energydensity is delivered to a part location, a void can remain in the part.The M290 printer was programmed to use relatively high energy parametersto produce spheres of keyhole anomalies. Groups of six spheres werearranged in a hexagonal pattern and ten regions with decreasing spherediameter were laid out at different heights in the build. The spherediameters ranged from 0.9 mm to 0.05 mm. Of the two keyhole parts, thepart at the bottom of the plate was selected for CT analysis because thein-process metrics show stronger signals at the programmed anomalies.The Metrology Center reported the anomalies for the keyhole part in twoSTL files. One part is for inclusions which was not included in theanalysis. The second part is for porosities, which are more appropriatefor describing the keyhole anomalies.

Tungsten Inclusion

FIG. 6 shows tessellations of the CT data for inclusion parts 225 withregions 223 corresponding to inclusion regions. When foreign particleswith high melting point contaminate a part, the particles can be fixedin the product which can result in crack initiation sites degrading partperformance. The M290 printer was paused before spreading powder atthree different layers. The build chamber was opened, and graduatedamounts of Tungsten powder were deposited in four parts. Of the fourtungsten inclusion parts, the second one from the top of the plate wasselected for CT analysis because the in-process metrics show strongersignals at the programmed anomalies.

Registration of Results

The CT results were reported in the STL files illustrated in theprevious section. The anomalies appear as tessellations around thedetected voids and inclusions. The coordinate system in the STL fileswas adjusted to be the same as the coordinate system used in the STLfiles specifying the build parameters.

The PrintRite3D® in-process metrics (e.g., TED, TED Sigma, TEP and TEPSigma) are recorded through on-axis photodetectors that closely followthe movements of the laser. Minor differences can occur throughdistortions of the optical system and through shrinkage and othermechanical strains introduced by cooling of the material and byseparation of the part from the build plate.

The ranges of the x, y, and z coordinates in the STL files wereextracted, and were matched against the x, y, and layer coordinates inthe TED metrics. The average of the differences was used as offsets onthe STL coordinates to fine tune the matching of the coordinate systems.

The tessellations representing the anomalies span several layers; theywere projected onto individual layers as polygons. Labels were generatedby filling layer grid cells with weighted sums calculated as the lengthsof the enclosing polygons, to emphasize the weight of larger voids.Points on the perimeters of the polygons were added in that were omittedin the polygon representation. These combined weights can be largepositive integers, so they were normalized with the function:

${\frac{2}{e^{- x} + 1} - 1},{{{where}\mspace{14mu} x} = {{label}\mspace{14mu}{value}}}$This resulted in label values that can be treated as probabilities inthe range 0.0 to 1.0.Printrite3D® ML Models

The features available for ML models within the Printrite3D system arethe in-process metrics from the build. The four primary metrics TED, TEDSigma, TEP, and TEP Sigma that are collected on grid cells with internalmetrics that are used to calculate TEP Sigma, namely the counts and thetep_sums. Since there could be inter-layer interactions, these sixfeatures were extended with the corresponding values on the previouslayer. During training of the models, the inclusion of these featuresand other potential candidates were evaluated by checking featureimportance reported by the models. One of skill in the art having thebenefit of this disclosure will appreciate that any in-process metricscould be used as input into the machine learning algorithm and thisdisclosure is in no way limited to the specific in-process metricsavailable within the Printrite3D system.

Ensemble methods that could be run in parallel on multiple processorswere investigated. In this example a random forest of classificationmachine learning model was chosen, however one of ordinary skill in theart having the benefit of this disclosure will appreciate that any othersuitable machine learning model could be used. One suitable model classis a random forest regressor. To improve the model, in some embodiments,it can be augmented by decision trees or other ensemble methods such asextra trees, bagging, or boosting. The grid cells in the layer in partsof a build amount to tens of millions of data points, so the ensemblemethods may also be suitable because of their performance at scale.

Methods were selected that could be run in parallel on multipleprocessors. For this embodiment, it was found that currentimplementations of the boosting methods were not fast enough. In thisexample embodiment, gradient boosting and bagging gave fairly goodperformance in terms of accuracy and speed, but not as good as randomforest or extremely randomized trees. In the end, it was determined thatrandom forest was more accurate than extremely randomized trees on ourdata, so the sklearn.ensemble.RandomForestClassifier was selected forthis example embodiment, however other models may also be suitable.

The ensemble of trees selected are controlled by the random_stateparameter, which then propagates new random states to each componentestimator. These component estimators are decision trees withindependent choices on subsets of the features to use when branching tofit a data set. The prediction at each point is computed as the averageof the predictions of the individual trees at that point. It was foundthat training with twice as many estimators as desired and then pruningthe weaker trees improved accuracy, however in other embodimentsdifferent techniques can be used.

To ameliorate over-training, in this embodiment 10-fold cross validationon the data was used. This means that the data set was partitionedrandomly into 10 equal folds, and for each test fold training wasperformed on the other 9 folds and evaluation was performed on theselected test fold. Other model parameters were also used for regulatingthe growth of the trees.

A pair of cost functions were developed to measure residuals in thepositive and negative directions. In this embodiment missing an anomalywas deemed more important than mis-flagging non-anomalous conditions, sothe negative residuals were weighted as 10 times more expensive than thepositive residuals to focus on the false negatives. The total cost isthen the sum of the positive residuals and the weighted negativeresiduals, and this was chosen as the primary metric when choosingmodels and tuning them. Note that these costs are unitless and do notreflect dollar cost.

Binary classification residuals are calculated as the differencesbetween binary labels and binary predictions, but since the labels areprobabilities, prediction probabilities were used and the residuals werecalculated as floating-point differences. Binary residual valuescollapse to −1, 0, and 1, but probability residuals cover the full rangeand can distinguish small and large differences.

In this example embodiment, the number of grid cells labeled anomalousis smaller than the number labeled normal, but least-squares fitting maybe used when the conditions are balanced. The normal data wassub-sampled and the anomalies with replacement were over-sampled tobring the counts into approximate balance, and then class weights werecalculated to account for any remainder. For example, when there are 100times more normals than anomalies, the process might sub-sample 20% ofthe normals and over-sample the anomalies 5 times, and then set theclass weight for anomalies as 4 times more important than normals tocomplete the balance.

FIG. 7 illustrates an example heat map 700 that can be used to diagnosehow well model prediction images compare to training labels. Thefour-color classification map shown in FIG. 7 can be used where greenindicates True Positive 705, white indicates True Negative 710, redindicates False Negative 715, and blue indicates False Positive 720.Since the labels take floating point values in the range of 0.0 to 1.0and the predictions represent the probabilities of a grid cell being ananomaly, heatmap 700 blurs the colors between the categories. One ofskill in the art having the benefit of this disclosure will appreciatethat other types of anomaly identification methods can be used and arewithin the scope of this disclosure.

Lack of Fusion Model

FIG. 8A shows the lack of fusion anomalies 805 in the CT visualization.The first machine learning model was trained with lack of fusion parts215 (see FIG. 2A). FIG. 7B shows the machine learning algorithmprediction diagnostic visualization using heatmap 700 of FIG. 7 . Asshown in FIG. 8B, the lack of fusion regions 805 are predominantly shownin Green corresponding to True Positive 705. The 10-fold crossvalidation was applied when calculating the negative mean squared errorand evaluated as the mean value of the ten numbers. For the costfunction, the cost values were normalized to compensate for the crossvalidation divisions. Following are the evaluation values for the lackof fusion model:

-   test neg_mean_squared_error mean: −0.010 (+/−0.0001)-   test total_cost mean: 364079.345 (+/−1535.2343)

Gas Porosity Model

FIG. 9A shows the gas porosity anomalies 905 in the CT visualization.The second machine learning model was trained on gas porosity parts 220(see FIG. 2 ) caused by anhydrous sodium acetate powder. FIG. 9B showsthe machine learning layer diagnostic visualization with heatmap 700 ofFIG. 7 . As shown in FIG. 9B, the regions of gas porosity 905 arepredominatley shown in Green corresponding to True Positive 705. Thefollowing are the 10-fold cross validation scores for the GP model:

-   test neg_mean_squared_error mean: −0.003 (+/−0.0000)-   test total_cost mean: 117800.665 (+/−526.8486)

Keyhole Model

FIG. 10A shows the keyhole anomalies 1005 in the CT visualization. Thethird machine learning model was trained with keyhole parts 210 (seeFIG. 2 ) caused by reduced power from the energy source. FIG. 10B showsthe ML prediction diagnostic visualization with the heatmap 700 of FIG.7 . As shown in FIG. 10B, the keyhole regions 1005 are predominantlyshown in Green corresponding to True Positive 705. The following are the10-fold cross validation scores for the GP model:

-   test neg_mean_squared_error mean: −0.011 (+/−0.0001)-   test total_cost mean: 413533.676 (+/−1545.1225)

Tungsten Model

FIG. 11A shows the tungsten inclusion anomalies 1105 in the CTvisualization. The fourth machine learning model was trained withinclusion parts 225 (see FIG. 2 ) caused by adding tungsten powder tothree layers. FIG. 11B shows the machine learning prediction diagnosticvisualization with heatmap 700 of FIG. 7 . As shown in FIG. 11B, theinclusion regions 1105 are predominantly shown in Green corresponding toTrue Positive 705. The following are the 10-fold cross validation scoresfor the W model:

-   test neg_mean_squared_error mean: −0.023 (+/−0.0001)-   test total_cost mean: 526917.446 (+/−2611.9907)    Adversarial Modeling

The models can be trained to recognize their anomaly types and avoidflagging other anomaly types. However, rejection of other anomaly typescan be enhanced by explicitly training against them. This section showshow to perform adversarial enhancement of the machine learning models.

Lack of Fusion Model

Table 1 shows the layer cost values of lack of fusion (LOF) modeltesting the parts with four different anomaly types. FIG. 12A shows thetest layer visualization with respect to keyhole 1200, FIG. 12B showsthe test layer visualization with respect to gas porosity 1205, FIG. 12Cshows the test layer visualization with respect to inclusions 1210. Thecost calculations are shown below in Table 1. The low cost value on thesame type (e.g., lack of fusion in this case) and the high cost value onthe other types imply that the model trained with the lack of fusiondata rejects the other anomalies and considers them false positives.

TABLE 1 LOF model Total Cost Positive Cost Negative Cost lack of fusion1186 68 1118 keyhole 2105 108 1997 gas porosity 3604 141 3463 tungsten24192 24 24169

In some embodiments another way to evaluate an adversarially trainedmodel is to visualize the predictions, which can be performed, forexample, in the PrintRite3D® user interface. FIG. 12D shows the LOFmodel prediction on a lack of fusion part. Lack of fusion regions 1225show up indicating that the model accurately predicts the lack of fusionanomaly. FIGS. 12E-12G show the LOF model predictions on the parts withkeyhole, gas porosity and tungsten anomalies, respectively, with eachprediction showing low probabilities of being lack of fusion anomalies.

FIG. 12H shows the prediction on PrintRite3D® visualization generated bythe LOF model. From this 3D tool, it can be seen that the LOF model isable to recognize the anomalies from another part with lack of fusionanomalies and ignore most of the other anomaly types on the build plate.

Gas Porosity Model

Table 2 shows the layer cost values of gas porosity (GP) model testingthe parts with four anomaly types. FIG. 13A shows the test layervisualization with respect to lack of fusion 1305, while FIGS. 13B and13C show the test layer visualization with respect to keyhole andtungsten inclusion, respectively. The low cost value on the same type(e.g., gas porosity in this case) and the high cost value on the othertypes imply that the model trained with the gas porosity data rejectsthe other anomalies and considers them false positives.

TABLE 2 GP model Total Cost Positive Cost Negative Cost gas porosity1288 77 1211 lack of fusion 3694 70 3625 keyhole 2025 6 2019 tungsten24343 0 24343

In some embodiments another way to evaluate an adversarially trainedmodel is to visualize the predictions, which can be performed, forexample, in the PrintRite3D® user interface. FIG. 13E shows the GP modelprediction on a gas porosity part. Gas porosity regions 1325 show upindicating that the model accurately predicts the gas porosity anomaly.FIGS. 13D, 13F and 13G show the model predictions on the parts with lackof fusion, keyhole, and tungsten inclusion anomalies, respectively, witheach prediction showing low probabilities of being gas porosityanomalies.

FIG. 13H shows the prediction on PrintRite3D® visualization generated bythe GP model. The anomalies appeared only in the part that was used fortraining. The anomalies did not show up in the other parts of the sametype, because the training part had been selected based on the strengthof its metrics.

Keyhole Model

Table 3 shows the layer cost values of key hole (KH) model testing theparts with four anomaly types. FIG. 14A shows the test layervisualization with respect to lack of fusion 1405, while FIGS. 14B and14C show the test layer visualization with respect to gas porosity, andtungsten anomalies, respectively. The low cost value on the same typeand the high cost value on the other types imply that the model trainedwith the keyhole data rejects the other anomalies and considers themfalse positives.

TABLE 3 KH model Total Cost Positive Cost Negative Cost keyhole 885 122762 lack of fusion 3677 79 3598 gas porosity 3544 54 3491 tungsten 243460 24346

In some embodiments another way to evaluate an adversarially trainedmodel is to visualize the predictions, which can be performed, forexample, in the PrintRite3D® user interface. FIG. 14D shows the KH modelprediction on a keyhole part. FIGS. 14E-14G show the model predictionson the parts with lack of fusion, gas porosity and tungsten anomalies,respectively. In the keyhole case (FIG. 14D), the keyhole anomalies 1405show up with high probabilities, whereas the lack of fusion, gasporosity and tungsten anomalies, FIGS. 14E-14G, respectively, have lowprobabilities of being keyhole anomalies.

FIG. 14H shows the prediction on PrintRite3D® visualization generated bythe KH model. From this tool, it can be seen that the KH model is ableto recognize only a few of the anomalies from another part with keyholeanomalies and ignore other anomaly types on the build plate.

Tungsten Model

Table 4 shows the layer cost values of the tungsten inclusion modeltesting the parts with four anomaly types. FIGS. 15A-15C show the testlayer visualization with respect to lack of fusion, gas porosity, andkeyhole anomalies, respectively. The low cost value on the same type andthe high cost value on the other types imply that the model trained withthe inclusion data rejects the other anomalies and considers them falsepositives.

TABLE 4 W model Total Cost Positive Cost Negative Cost tungsten 7294 7256569 lack of fusion 3640 3 3636 gas porosity 3551 54 3498 keyhole 2019 12018

In some embodiments another way to evaluate an adversarially trainedmodel is to visualize the predictions, which can be performed, forexample, in the PrintRite3D® user interface. FIG. 15E shows the tungsteninclusion model prediction on a different tungsten inclusion part andthe prediction colormap. FIGS. 15D, 15F and 15G show the modelpredictions on the parts with lack of fusion, gas porosity and keyholeanomalies, respectively. In the tungsten inclusion case (FIG. 15E), thetungsten inclusion anomalies 1525 show up with high probabilities. Incontrast, the lack of fusion, gas porosity and keyhole anomalies havelow probabilities of being tungsten inclusion anomalies.

FIG. 15H shows the prediction on PrintRite3D® visualization generated bythe tungsten inclusion model. The anomalies appeared only in the partthat was used for training in the layers where tungsten particles wereapplied. The anomalies showed up in some of the other parts of the sametype, because the training part had been selected based on the strengthof its metrics. The anomalies also showed up in the support structure ofthe trained tungsten part and a few other parts.

Conclusion

It has been demonstrated that machine learning models can be trained torecognize at least four different common types of anomalies: lack offusion, gas porosity, keyhole, and tungsten inclusions. It will beappreciated by one of skill in the art having the benefit of thisdisclosure that other anomaly types can be detected and identified usinga similar methodology. The models use PrintRite3D® metrics includingTED, TED SIGMA, TEP and TEP SIGMA as features and the CT data as labels,however other embodiments can use other metrics. The predictions show astrong correlation with the CT labels. The models were trained both toagree with the labels for the anomaly type and to reject anomalies ofthe other three types. This resulted in improved discrimination in thepredictions of each anomaly type.

The specific example embodiment described herein focused on producingIPQM^(P) metrics for each anomaly type, however other embodiments canuse different metrics. Prediction metrics like these can make itpossible to close the loop on controlling the build process. Morespecifically, anomalies can be automatically repaired or brought to theattention of production managers who can decide whether or not tointerrupt the build process.

In some embodiments, the machine learning models access the productionprocess and classify the anomalous signals into the various types. Suchan all-in-one model could be built on the techniques developed in theembodiment described herein, however in other embodiments other suitablemachine learning techniques can be used.

Example Additive Manufacturing Apparatus

FIG. 16 shows an exemplary additive manufacturing system 1600. As shownin FIG. 16 , additive manufacturing system 1600 includes a laser as anenergy source 1605, however other embodiments may use an electron beamor other suitable energy source. Energy source 1605 emits a laser beam1610 which passes through a partially reflective mirror 1615 and entersa scanning and focusing system 1620 which then projects the beam to aregion 1625 on build plane 1630. In some embodiments, build plane 1630is a powder bed of metallic powder 1635. Electromagnetic energy 1640 isemitted from region 1625 due to high material temperatures andemissivity properties of the materials receiving being irradiated bylaser beam 1610. Laser beam 1610 fuses metallic powder 1635 into a part1645 that is built layer by layer.

In some embodiments, the scanning and focusing system 1620 can beconfigured to collect some of the electromagnetic energy 1640 emittedfrom region 1625. In some embodiments, a melt pool and luminous plumecan cooperatively emit blackbody radiation from within region 1625. Themelt pool is the result of powdered metal 1635 liquefying due to theenergy imparted by laser beam 1610 and is responsible for the emissionof most of the electromagnetic energy 1640 being reflected back towardfocusing system 1620. The luminous plume results from vaporization ofportions of the powdered metal 1635. The partially reflective mirror1615 can reflect most of the electromagnetic energy 1640 received byfocusing system 1620. This reflected energy is indicated on FIG. 16 asoptical energy 1655. Optical energy 1655 may be interrogated by on-axisoptical sensors 1650. Each of the on-axis optical sensors 1650 receive aportion of optical energy 1655 through mirrors 1660 a and 1660 b. Insome embodiments, mirrors 1660 can be configured to reflect onlywavelengths λ₁ and λ₂, respectively. In some embodiments, opticalsensors 1650 receive a total of 80-90% of the light reflected throughthe optics train.

Optical sensors 1650 can also include notch filters that are configuredto block any light outside of respective wavelengths λ₁ and λ₂. Thirdoptical sensor 1650 c can be configured to receive light from partiallyreflective mirror 1660 c. In some embodiments, optical sensors 1650 aand 1650 b can be covered by notch filters while third optical sensor1650 c can be configured to measure a much larger range of wavelengths.In some embodiments, optical sensor 1650 a or 1650 b can be replacedwith a spectrometer configured to perform an initial characterization ofa blackbody radiation curve associated with a batch of powder being usedto perform an additive manufacturing process. This characterization canthen be used to determine how the wavelength filters of optical sensors1650 a and 1650 b are configured to be offset and avoid any spectralpeaks associated with the black body curve characterized by thespectrometer. This characterization is performed prior to a fulladditive manufacturing operation being carried out.

In some embodiments third optical sensor 1650 c is configured to measurethermal energy density. The thermal energy density is sensitive tochanges in process parameters such as, for example, energy source power,energy source speed, and hatch spacing.

Examples of on-axis optical sensors 1650 include but are not limited tophoto to electrical signal transducers (i.e. photodetectors) such aspyrometers and photodiodes. Optical sensors 1650 can also includespectrometers, and low or high speed cameras that operate in thevisible, ultraviolet, or the infrared frequency spectrum. The on-axisoptical sensors 1650 are in a frame of reference which moves with laserbeam 1610, i.e., they see all regions that are touched by the laser beamand are able to collect electromagnetic energy 1640 from all regions ofbuild plane 1630 touched as the laser beam 1610 scans across build plane1630. Because the optical energy 1640 collected by the scanning andfocusing system 1620 travels a path that is near parallel to the laserbeam, sensors 1650 can be considered on-axis sensors.

In some embodiments, additive manufacturing system 1600 can include oneor more off-axis sensors 1665 that are in a stationary frame ofreference with respect to laser beam 1610. Additionally, there could becontact sensors on a recoater arm 1670 configured to spread metallicpowder 1635 across build plane 1630. These sensors could beaccelerometers, vibration sensors, etc. Lastly, there could be othertypes of sensors such as thermocouples to measure macro thermal fieldsor could include acoustic emission sensors which could detect crackingand other metallurgical phenomena occurring in part 1645 as it is beingbuilt.

In some embodiments, a computing system 1675, including one or moreprocessors 1680, computer readable medium 1685, and an I/O interface1690, is provided and coupled to suitable system components of theadditive manufacturing system in order to collect data from the varioussensors. Data received by computing system 1675 can include in-processraw sensor data and/or reduced order sensor data. One or more processors1680 can use in-process raw sensor data and/or reduced order sensor datato determine laser 1610 power and control information, includingcoordinates in relation to build plane 1630. In other embodiments,computing system 1675, including one or more processors 1680, computerreadable medium 1685, and an I/O interface 1690, can provide for controlof the various system components. Computing system 1675 can send,receive, and monitor control information associated with laser 1605,build plane 1630, and other associated components and sensors.

One or more processors 1680 can be used to perform calculations usingthe data collected by the various sensors to generate in-process qualitymetrics (e.g., TED, TED Sigma, TEP, TEP Sigma). In some embodiments,data generated by on-axis optical sensors 1650 can be used to determinethermal energy density (TED) and/or TEP during the build process.Control information associated with movement of the energy source 1605across the build plane 1630 can be received by one or more processors1680. One or more processors 1680 can then use the control informationto correlate data from on-axis optical sensor(s) 1650 and/or off-axisoptical sensor(s) 1665 with a corresponding location. This correlateddata can then be combined to calculate thermal energy density. In someembodiments, the thermal energy density and/or other metrics can be usedby one or more processors 1680 to generate control signals for processparameters, for example, laser power, laser speed, hatch spacing, andother process parameters in response to the thermal energy density orother metrics. As described herein, one or more processors 1680 may usethe one or more process parameters as inputs into one or more trainedmachine learning algorithms to detect and identify defects in part 1645real-time. In this way, a problem that might otherwise ruin a productionpart can be ameliorated. In embodiments where multiple parts are beinggenerated at once, prompt corrections to the process parameters inresponse to metrics falling outside desired ranges can prevent adjacentparts from receiving too much or too little energy from the energysource.

In some embodiments, I/O interface 1690 can be configured to transmitdata collected to a remote location. I/O interface 1690 can beconfigured to receive data from a remote location. The data received caninclude baseline datasets, historical data, post-process inspectiondata, and classifier data. The remote computing system can calculatein-process quality metrics using the data transmitted by the additivemanufacturing system. The remote computing system can transmitinformation to I/O interface 1690 in response to in-process qualitymetrics and/or detection of defects. It should be noted that the sensorsdescribed in conjunction with FIG. 16 can be used in the described waysto characterize performance of any additive manufacturing processinvolving sequential material build up.

While the embodiments described herein have used data generated byoptical sensors to determine the thermal energy density, the embodimentsdescribed herein may be implemented using data generated by sensors thatmeasure other manifestations of in-process physical variables. Sensorsthat measure manifestations of in-process physical variables include,for example, force and vibration sensors, contact thermal sensors,non-contact thermal sensors, ultrasonic sensors, and eddy currentsensors. It will be apparent to one of ordinary skill in the art thatmany modifications and variations are possible in view of the aboveteachings.

In some embodiments TEP can include the steps of identifying spectralpeaks associated with material properties of a batch of powder andselecting a first wavelength and a second wavelength spaced apart fromthe first wavelength, the first wavelength and the second wavelengthbeing offset from the identified spectral peaks. A plurality of scans ofan energy source can be generated across a layer of the batch of powderdisposed upon a build plane during an additive manufacturing operation.An amount of energy radiated from the build plane can be measured at thefirst wavelength and an amount of energy radiated from the build planecan be measured at the second wavelength. Variations in temperature ofan area of the build plane traversed by the plurality of scans can bedetermined based upon a ratio of energy radiated at the first wavelengthto energy radiated at the second wavelength. In some embodiments it canbe determined that the variations in temperature are outside a thresholdrange of values, and thereafter, adjusting subsequent scans of theenergy source across or proximate the area of the build plane.

In various embodiments the additive manufacturing method above furthercomprises determining the area of the build plane traversed by:determining a start point of a first scan of the plurality of scans anddetermining an end point of the first scan. Further, a length of thefirst scan can be determined by calculating a distance between the startpoint and the end point.

In some embodiments the additive manufacturing method above furthercomprises: mapping a thermal energy density to locations within a partbeing formed by the additive manufacturing operation by receiving energysource drive signal data indicating a path of the energy source acrossthe build plane, and determining a location of each of the plurality ofscans using the energy source drive signal data.

In some embodiments TED includes an additive manufacturing method,comprising generating a plurality of scans of an energy source across abuild plane and measuring an amount of energy radiated from the buildplane during each of the plurality of scans using an optical sensormonitoring the build plane. An area of the build plane traversed duringone or more scans of the plurality of scans of the energy source isdetermined. A thermal energy density for the area of the build planetraversed by the one or more scans of the plurality of scans isdetermined based upon the amount of energy radiated and the area of thebuild plane traversed by the plurality of scans. The thermal energydensity is mapped to one or more locations of the build plane. In someembodiments it can be determined that the thermal energy density ischaracterized by a density outside a range of density values, andthereafter, adjusting subsequent scans of the energy source across orproximate the one or more locations of the build plane.

In some embodiments the additive manufacturing method described abovefurther includes wherein determining that the thermal energy density ischaracterized by a density outside a range of density values furthercomprises, receiving a baseline and determining one of more thermalenergy density scan values are substantially different than thebaseline. At least one of a graph and a point cloud can be generated asan output.

FIG. 17 shows an expanded block diagram of an example computing system1675. In some embodiments, example computing system can implement any orall of the functions, behaviors, and capabilities described herein asbeing performed by a computing system or processor, as well as otherfunctions, behaviors, and capabilities not expressly described.Computing system 1675 can include one or more processors 1680, computerreadable medium 1685, user interface 1705 and network interface 1710.Computing system 1675 can also include other components (not explicitlyshown) such as a battery, power controllers, and other componentsoperable to provide various enhanced capabilities. In variousembodiments, computing system 1675 can be implemented in a desktopcomputer, laptop computer, tablet computer, smart phone, other mobilephone, wearable computing device, or other systems having any desiredform factor. Further, as noted above, computing system 1675 can beimplemented partly in a base station and partly in a mobile unit thatcommunicates with the base station and provides a user interface.

Computer readable medium 1685 can be implemented, e.g., using disk,flash memory, or any other non-transitory storage medium, or acombination of media, and can include volatile and/or non-volatilemedia. In some embodiments, computer readable medium 1685 can store oneor more application and/or operating system programs to be executed byone or more processors 1680, including programs to implement variousoperations described above as being performed by a host. For example,computer readable medium 1685 can store a uniform host application thatcan read a peripheral device description record and generate a graphicaluser interface for controlling the peripheral device based oninformation therein. Computer readable medium 1685 can also storeprogram code executable to communicate with a transceiver 1715. AlthoughFIG. 17 illustrates transceiver 1715 as a subsystem of computing system1675 it is understood that transceiver 1715 may be a dongle that isplugged into and electrically coupled with computing system 1675 forexample to enable machine learning algorithms or in-process parameters(e.g., TEP, TEP Sigma, TED or TED Sigma) to be executed. In someembodiments, portions (or all) of the host functionality describedherein can be implemented in operating system programs rather thanapplications. In some embodiments, computer readable medium 1685 canalso store apps designed for specific accessories or specific categoriesof accessories (e.g., an IP camera app to manage an IP camera peripheraldevice or a security app to interact with door lock accessories).

User interface 1705 can include input devices such as a touch pad, touchscreen, scroll wheel, click wheel, dial, button, switch, keypad,microphone 1720, or the like, as well as output devices such as a videoscreen, indicator lights, speakers, headphone jacks, or the like,together with supporting electronics (e.g., digital-to-analog oranalog-to-digital converters, signal processors, or the like). A usercan operate input devices of user interface 1705 to invoke thefunctionality of computing system 1675.

One or more processors 1680 can be implemented as one or more integratedcircuits, e.g., one or more single-core or multi-core microprocessors ormicrocontrollers, examples of which are known in the art. In operation,one or more processors 1680 can control the operation of computingsystem 1685 and/or additive manufacturing system 1600 (see FIG. 16 ). Invarious embodiments, one or more processors 1680 can execute a varietyof programs in response to program code and can maintain multipleconcurrently executing programs or processes. At any given time, some orall of the program code to be executed can be resident in one or moreprocessors 1680 and/or in storage media such as computer readable medium1685.

Further, while a computing system is described herein with reference toparticular blocks, it is to be understood that these blocks are definedfor convenience of description and are not intended to imply aparticular physical arrangement of component parts. Further, the blocksneed not correspond to physically distinct components. Blocks can beconfigured to perform various operations, e.g., by programming aprocessor or providing appropriate control circuitry, and various blocksmight or might not be reconfigurable depending on how the initialconfiguration is obtained. Embodiments of the present disclosure can berealized in a variety of apparatus including electronic devicesimplemented using any combination of circuitry and software.

Computing systems and accessories described herein can be implemented inelectronic devices that can be of generally conventional design. Suchdevices can be adapted to communicate using a uniform peripheral deviceprotocol that supports command-and-control operations by which a host (afirst electronic device) can control operation of a peripheral device (asecond electronic device). In some instances, a device can combinefeatures or aspects of a host and a peripheral device, e.g., in the caseof a proxy as described above.

It will be appreciated that the system configurations and componentsdescribed herein are illustrative and that variations and modificationsare possible. It is to be understood that an implementation of computingsystem 1675 can perform all operations described above as beingperformed by a media access device and that an implementation of aperipheral device can perform any or all operations described above asbeing performed by a peripheral device. A proxy, bridge, tunnel, orcoordinator can combine components of computing system 1675 and aperipheral device, using the same hardware or different hardware asdesired. The media access device and/or peripheral device may have othercapabilities not specifically described herein (e.g., mobile phone,global positioning system (GPS), broadband data communication, Internetconnectivity, etc.). Depending on implementation, the devices caninteroperate to provide any functionality supported by either (or both)devices or to provide functionality that is partly implemented in eachdevice. In some embodiments, a particular peripheral device can havesome functionality that is not accessible or invocable via a particularmedia access device but is accessible via another host or by interactingdirectly with the peripheral device.

Further, while the media access device and peripheral device aredescribed herein with reference to particular blocks, it is to beunderstood that these blocks are defined for convenience of descriptionand are not intended to imply a particular physical arrangement ofcomponent parts. Further, the blocks need not correspond to physicallydistinct components. Blocks can be configured to perform variousoperations, e.g., by programming a processor or providing appropriatecontrol circuitry, and various blocks might or might not bereconfigurable depending on how the initial configuration is obtained.Embodiments of the present disclosure can be realized in a variety ofapparatus including electronic devices implemented using any combinationof circuitry and software.

Various features described herein, e.g., methods, apparatus,computer-readable media and the like, can be realized using anycombination of dedicated components and/or programmable processorsand/or other programmable devices. The various processes describedherein can be implemented on the same processor or different processorsin any combination. Where components are described as being configuredto perform certain operations, such configuration can be accomplished,e.g., by designing electronic circuits to perform the operation, byprogramming programmable electronic circuits (such as microprocessors)to perform the operation, or any combination thereof. Further, while theembodiments described above may make reference to specific hardware andsoftware components, those skilled in the art will appreciate thatdifferent combinations of hardware and/or software components may alsobe used and that particular operations described as being implemented inhardware might also be implemented in software or vice versa.

Computer programs incorporating various features described herein may beencoded and stored on various computer readable storage media; suitablemedia include magnetic disk or tape, optical storage media such ascompact disk (CD) or DVD (digital versatile disk), flash memory, andother non-transitory media. Computer readable media encoded with theprogram code may be packaged with a compatible electronic device, or theprogram code may be provided separately from electronic devices (e.g.,via Internet download or as a separately packaged computer-readablestorage medium).

Numerous specific details are set forth herein to provide a thoroughunderstanding of the claimed subject matter. However, those skilled inthe art will understand that the claimed subject matter may be practicedwithout these specific details. In other instances, methods,apparatuses, or systems that would be known by one of ordinary skillhave not been described in detail so as not to obscure claimed subjectmatter. The various embodiments illustrated and described are providedmerely as examples to illustrate various features of the claims.However, features shown and described with respect to any givenembodiment are not necessarily limited to the associated embodiment andmay be used or combined with other embodiments that are shown anddescribed. Further, the claims are not intended to be limited by any oneexample embodiment.

While the present subject matter has been described in detail withrespect to specific embodiments thereof, it will be appreciated thatthose skilled in the art, upon attaining an understanding of theforegoing may readily produce alterations to, variations of, andequivalents to such embodiments. Accordingly, it should be understoodthat the present disclosure has been presented for purposes of examplerather than limitation, and does not preclude inclusion of suchmodifications, variations, and/or additions to the present subjectmatter as would be readily apparent to one of ordinary skill in the art.Indeed, the methods and systems described herein may be embodied in avariety of other forms; furthermore, various omissions, substitutionsand changes in the form of the methods and systems described herein maybe made without departing from the spirit of the present disclosure. Theaccompanying claims and their equivalents are intended to cover suchforms or modifications as would fall within the scope and spirit of thepresent disclosure.

Although the present disclosure provides certain example embodiments andapplications, other embodiments that are apparent to those of ordinaryskill in the art, including embodiments which do not provide all of thefeatures and advantages set forth herein, are also within the scope ofthis disclosure. Accordingly, the scope of the present disclosure isintended to be defined only by reference to the appended claims.

Unless specifically stated otherwise, it is appreciated that throughoutthis specification discussions utilizing terms such as “processing,”“computing,” “calculating,” “determining,” and “identifying” or the likerefer to actions or processes of a computing device, such as one or morecomputers or a similar electronic computing device or devices, thatmanipulate or transform data represented as physical electronic ormagnetic quantities within memories, registers, or other informationstorage devices, transmission devices, or display devices of thecomputing platform.

The system or systems discussed herein are not limited to any particularhardware architecture or configuration. A computing device can includeany suitable arrangement of components that provide a result conditionedon one or more inputs. Suitable computing devices include multi-purposemicroprocessor-based computer systems accessing stored software thatprograms or configures the computing system from a general purposecomputing apparatus to a specialized computing apparatus implementingone or more embodiments of the present subject matter. Any suitableprogramming, scripting, or other type of language or combinations oflanguages may be used to implement the teachings contained herein insoftware to be used in programming or configuring a computing device.

Embodiments of the methods disclosed herein may be performed in theoperation of such computing devices. The order of the blocks presentedin the examples above can be varied—for example, blocks can bere-ordered, combined, and/or broken into sub-blocks. Certain blocks orprocesses can be performed in parallel.

Conditional language used herein, such as, among others, “can,” “could,”“might,” “may,” “e.g.,” and the like, unless specifically statedotherwise, or otherwise understood within the context as used, isgenerally intended to convey that certain examples include, while otherexamples do not include, certain features, elements, and/or steps. Thus,such conditional language is not generally intended to imply thatfeatures, elements and/or steps are in any way required for one or moreexamples or that one or more examples necessarily include logic fordeciding, with or without author input or prompting, whether thesefeatures, elements and/or steps are included or are to be performed inany particular example.

The terms “comprising,” “including,” “having,” and the like aresynonymous and are used inclusively, in an open-ended fashion, and donot exclude additional elements, features, acts, operations, and soforth. Also, the term “or” is used in its inclusive sense (and not inits exclusive sense) so that when used, for example, to connect a listof elements, the term “or” means one, some, or all of the elements inthe list. The use of “adapted to” or “configured to” herein is meant asopen and inclusive language that does not foreclose devices adapted toor configured to perform additional tasks or steps. Additionally, theuse of “based on” is meant to be open and inclusive, in that a process,step, calculation, or other action “based on” one or more recitedconditions or values may, in practice, be based on additional conditionsor values beyond those recited. Similarly, the use of “based at least inpart on” is meant to be open and inclusive, in that a process, step,calculation, or other action “based at least in part on” one or morerecited conditions or values may, in practice, be based on additionalconditions or values beyond those recited. Headings, lists, andnumbering included herein are for ease of explanation only and are notmeant to be limiting.

The various features and processes described above may be usedindependently of one another, or may be combined in various ways. Allpossible combinations and sub-combinations are intended to fall withinthe scope of the present disclosure. In addition, certain method orprocess blocks may be omitted in some embodiments. The methods andprocesses described herein are also not limited to any particularsequence, and the blocks or states relating thereto can be performed inother sequences that are appropriate. For example, described blocks orstates may be performed in an order other than that specificallydisclosed, or multiple blocks or states may be combined in a singleblock or state. The example blocks or states may be performed in serial,in parallel, or in some other manner. Blocks or states may be added toor removed from the disclosed examples. Similarly, the example systemsand components described herein may be configured differently thandescribed. For example, elements may be added to, removed from, orrearranged compared to the disclosed examples.

What is claimed is:
 1. An additive manufacturing system comprising: a recoater arm arranged to distribute layer of metallic powder across a build plane; a power source arranged to emit a beam of energy at the build plane and fuse the metallic powder into a portion of a part; and a processor configured to: access a part geometry for the part; steer the beam of energy across the build plane based on the part geometry; receive data generated by one or more sensors that detect electromagnetic energy emitted from the build plane when the beam of energy fuses the metallic powder; convert the received data into one or more parameters that indicate one or more conditions at the build plane while the beam of energy fuses the metallic powder; select a machine learning algorithm based on at least in part on the part geometry; use the one or more parameters as input into the machine learning algorithm to generate an output from the machine learning algorithm; and use the output of the machine learning algorithm to detect one or more defects in the fused metallic powder.
 2. The additive manufacturing system of claim 1 wherein the machine learning algorithm is further configured to determine a type of the one or more defects.
 3. The additive manufacturing system of claim 2 wherein the type of the one or more defects includes at least one of a lack of fusion defect, a porosity defect or an inclusion defect.
 4. The additive manufacturing system of claim 1 wherein the one or more parameters includes determining a thermal emission density (TED) that includes measuring an amount of energy radiated from the build plane during one or more scans and determining area of the build plane traversed during the one or more scans.
 5. The additive manufacturing system of claim 1 wherein the one or more parameters includes identifying spectral peaks associated with material properties of a batch of powder and selecting a first wavelength and a second wavelength spaced apart from the first wavelength, and determining an amount of energy radiated from the build plane based upon a ratio of energy radiated at the first wavelength to energy radiated at the second wavelength.
 6. The additive manufacturing system of claim 1 wherein the machine learning algorithm includes one or more training parameters based on a known-defective part.
 7. The additive manufacturing system of claim 6 wherein the one or more training parameters are derived from a known-defective part having at least one void.
 8. The additive manufacturing system of claim 6 wherein the one or more training parameters are derived from a known-defective part having at least one inclusion.
 9. The additive manufacturing system of claim 1 wherein the one or more sensors includes an on-axis photodetector.
 10. The additive manufacturing system of claim 1 wherein the detecting one or more defects further comprises a processor configured to determine whether the defect can be remedied.
 11. The additive manufacturing system of claim 1 wherein the part geometry is received as a stereolithography (STL) file. 